Method for converting tone of chest x-ray image, storage medium, image tone conversion apparatus, server apparatus, and conversion method

ABSTRACT

A method for converting tone of a chest X-ray image includes obtaining a target chest X-ray image, detecting, in the target chest X-ray image using a model obtained as a result of machine learning, a structure including a linear structure formed of a first linear area that has been drawn by projecting anatomical structures whose X-ray transmittances are different from each other or a second linear area drawn by projecting an anatomical structure including a wall of a trachea, a wall of a bronchus, or a hair line, extracting a pixel group corresponding to a neighboring area of the structure, generating a contrast conversion expression for histogram equalization using a histogram of the pixel group, and converting luminance of each pixel value in entirety of the target chest X-ray image using the contrast conversion expression.

BACKGROUND 1. Technical Field

The present disclosure relates to a technique for processing medicalimages and, more specifically, to a technique for converting tone of achest X-ray image.

2. Description of the Related Art

Costs of devices for capturing chest X-ray images and costs of capturingchest X-ray images are especially low among medical images, and suchdevices are widely used. Chest X-ray images, therefore, are a firstchoice for making diagnoses of chest diseases. In chest X-ray images,however, anatomical structures overlap one another in a depth direction.For this reason, interpretation is difficult, and there are problemsthat lesions can be overlooked and that computer tomography is performedwithout much consideration.

An X-ray image capture apparatus usually obtains a chest X-ray image asa fine-gradation (e.g., 10 to 14 bits) digital image. When the obtainedchest X-ray image is displayed on a monitor, however, the chest X-rayimage is subjected to tone compression to achieve coarser gradation(e.g., 8 to 12 bits) and displayed. The tone compression is performedalong with contrast conversion such as gamma correction so thatimportant tones in the image are saved. In order to make interpretationas easy as possible, it is important to perform tone compression suchthat information in an area important in making a diagnosis based on thechest X-ray image does not deteriorate.

International Publication No. 2015/174206 has proposed a technique forconverting tone capable of displaying a desired area with desired levelsof contrast and density while maintaining the amount of information of achest X-ray image. In the technique described in InternationalPublication No. 2015/174206, a range of pixel values in a broad areasuch as a lung field or a mediastinum is estimated from a pixel valuehistogram of a chest X-ray image, and a control point of a gamma curveis determined on the basis of a result of the estimation.

SUMMARY

In the technique described in International Publication No. 2015/174206,for example, a gamma curve suitable for a lung field or a mediastinum,for example, can be used. Because the example of the related art doesnot necessarily improve a level of contrast in an area important inmaking a diagnosis based on a chest X-ray image, however, furtherimprovements are required.

In one general aspect, the techniques disclosed here feature a methodfor converting tone of a chest X-ray image, the method being performedby a computer of an image tone conversion apparatus that converts toneof a target chest X-ray image, which is a chest X-ray image to beinterpreted, the method including obtaining the target chest X-rayimage, detecting, in the target chest X-ray image using a model obtainedas a result of machine learning, a structure including a linearstructure formed of a first linear area that has been drawn byprojecting anatomical structures whose X-ray transmittances aredifferent from each other or a second linear area drawn by projecting ananatomical structure including a wall of a trachea, a wall of abronchus, or a hair line, extracting a pixel group corresponding to aneighboring area of the structure, generating a contrast conversionexpression for histogram equalization using a histogram of the pixelgroup, and converting luminance of each pixel value in entirety of thetarget chest X-ray image using the contrast conversion expression.

The above aspect achieves further improvements.

It should be noted that this general or specific aspect may beimplemented as an apparatus, a system, an integrated circuit, a computerprogram, a computer-readable storage medium, or any selectivecombination thereof. The computer-readable storage medium may be anonvolatile storage medium such as a compact disc read-only memory(CD-ROM).

Additional benefits and advantages of the disclosed embodiments willbecome apparent from the specification and drawings. The benefits and/oradvantages may be individually obtained by the various embodiments andfeatures of the specification and drawings, which need not all beprovided in order to obtain one or more of such benefits and/oradvantages.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram illustrating an image tone conversionapparatus according to a first embodiment;

FIG. 2 is a block diagram illustrating a network configuration in amedical facility according to the first embodiment;

FIG. 3 is a flowchart according to the first embodiment;

FIG. 4A is a diagram illustrating a chest X-ray image including a shadowin the descending aorta;

FIG. 4B is a diagram illustrating a mask image of the shadow in thedescending aorta;

FIG. 4C is a diagram illustrating an image obtained by superimposing themask image upon the chest X-ray image;

FIG. 5A is a diagram illustrating a chest X-ray image including a shadowin the right dorsal diaphragm;

FIG. 5B is a diagram illustrating a mask image of a shadow in the rightdorsal diaphragm;

FIG. 5C is a diagram illustrating an image obtained by superimposing themask image upon the chest X-ray image;

FIG. 6A is a diagram illustrating a chest X-ray image including thefirst thoracic vertebra;

FIG. 6B is a diagram illustrating a mask image of the first thoracicvertebra;

FIG. 6C is a diagram illustrating an image obtained by superimposing themask image upon the chest X-ray image;

FIG. 7 is a diagram schematically illustrating the architecture ofU-Net;

FIG. 8A is a diagram schematically illustrating an example of a linearstructure;

FIG. 8B is a diagram schematically illustrating an example of aneighboring area of the linear structure illustrated in FIG. 8A;

FIG. 9A is a diagram schematically illustrating an example of an areastructure;

FIG. 9B is a diagram schematically illustrating an example of aneighboring area of the area structure illustrated in FIG. 9A;

FIG. 10 is a diagram schematically illustrating an example of a toneconversion lookup table (LUT);

FIG. 11A is a diagram schematically illustrating another example of theneighboring area of the linear structure illustrated in FIG. 8A;

FIG. 11B is a diagram schematically illustrating another example of theneighboring area of the area structure illustrated in FIG. 9A;

FIG. 12 is a block diagram illustrating the configuration of an imagetone conversion apparatus according to a second embodiment;

FIG. 13 is a flowchart illustrating a process for detecting structuresaccording to the second embodiment;

FIG. 14 is a diagram schematically illustrating resolution information;

FIG. 15 is a diagram schematically illustrating steps illustrated inFIG. 13;

FIG. 16 is a block diagram illustrating the configuration of an imagetone conversion apparatus according to a third embodiment;

FIG. 17 is a flowchart according to the third embodiment;

FIG. 18 is a block diagram illustrating a network configuration in amedical facility according to a fourth embodiment; and

FIG. 19 is a diagram illustrating a reason for expanding a contour by acertain number of pixels,

DETAILED DESCRIPTION Underlying Knowledge Forming Basis of PresentDisclosure

With the technique described in International Publication No.2015/174206, a gamma curve suitable for a lung field or a mediastinum,for example, can be used. The present inventor, however, has found thatan area in a chest X-ray image that is smaller than a lung field or amediastinum and whose range of shades is smaller than that of the lungfield or the mediastinum can be sometimes important in making adiagnosis. The technique described in International Publication No.2015/174206 does not necessarily improve a level of contrast in such anarea.

The present inventor has arrived at the following aspects, in which alevel of contrast in a small area in a chest X-ray image whose range ofshades is small (e.g., a linear structure that will be described later)and that is important in making a diagnosis based on the chest X-rayimage can be improved.

A first aspect of the present disclosure is

-   -   a method for converting tone of a chest X-ray image, the method        being performed by a computer of an image tone conversion        apparatus that converts tone of a target chest X-ray image,        which is a chest X-ray image to be interpreted, the method        including:    -   obtaining the target chest X-ray image;    -   detecting, in the target chest X-ray image using a model        obtained as a result of machine learning, a structure including        a linear structure formed of a first linear area that has been        drawn by projecting anatomical structures whose X-ray        transmittances are different from each other or a second linear        area drawn by projecting an anatomical structure including a        wall of a trachea, a wall of a bronchus, or a hair line;    -   extracting a pixel group corresponding to a neighboring area of        the structure;    -   generating a contrast conversion expression for histogram        equalization using a histogram of the pixel group; and    -   converting luminance of each pixel value in entirety of the        target chest X-ray image using the contrast conversion        expression.

A second aspect of the present disclosure is

-   -   a storage medium storing a program for causing a computer of an        image tone conversion apparatus that converts tone of a target        chest X-ray image, which is a chest X-ray image to be        interpreted, to perform a process, the storage medium being        nonvolatile and computer-readable, the process including:    -   obtaining the target chest X-ray image;    -   detecting, in the target chest X-ray image using a model        obtained as a result of machine learning, a structure including        a linear structure formed of a first linear area that has been        drawn by projecting anatomical structures whose X-ray        transmittances are different from each other or a second linear        area drawn by projecting an anatomical structure including a        wall of a trachea, a wall of a bronchus, or a hair line;    -   extracting a pixel group corresponding to a neighboring area of        the structure;    -   generating a contrast conversion expression for histogram        equalization using a histogram of the pixel group; and    -   converting luminance of each pixel value in entirety of the        target chest X-ray image using the contrast conversion        expression.

A third aspect of the present disclosure is

-   -   an image tone conversion apparatus including:    -   an obtainer that obtains a target chest X-ray image, which is a        chest X-ray image to be interpreted;    -   a detector that detects, in the target chest X-ray image using a        model obtained as a result of machine learning, a structure        including a linear structure formed of a first linear area that        has been drawn by projecting anatomical structures whose X-ray        transmittances are different from each other or a second linear        area drawn by projecting an anatomical structure including a        wall of a trachea, a wall of a bronchus, or a hair line;    -   an extractor that extracts a pixel group corresponding to a        neighboring area of the structure;    -   an equalizer that generates a contrast conversion expression for        histogram equalization using a histogram of the pixel group; and    -   a luminance converter that converts luminance of each pixel        value in entirety of the target chest X-ray image using the        contrast conversion expression.

In the first to third aspects, a structure including a linear structureformed of a first linear area that has been drawn by projectinganatomical structures whose X-ray transmittances are different from eachother or a second linear area drawn by projecting an anatomicalstructure including a wall of a trachea, a wall of a bronchus, or a hairline is detected in a target chest X-ray image, which is a chest X-rayimage to be interpreted, using a model obtained as a result of machinelearning. A pixel group corresponding to a neighboring area of thedetected structure is extracted. A contrast conversion expression forhistogram equalization is generated using a histogram of the extractedpixel group. Luminance of each pixel value in the entirety of the targetchest X-ray image is converted using the generated contrast conversionexpression. According to the first to third aspects, therefore, a levelof contrast in a neighboring area of a structure can be improved withoutbeing affected by pixels having pixel values whose frequencies are high.

In the first aspect, for example,

-   -   the model obtained as a result of the machine learning may be a        model subjected to the machine learning such that the structure        is detected in a learning chest X-ray image, which is a chest        X-ray image in a normal state, using a neural network that        performs prediction in units of pixels.

In this aspect, a structure is detected using a model subjected tomachine learning such that a structure is detected in a learning chestX-ray image, which is a chest X-ray image in a normal state, using aneural network that performs prediction in units of pixels, Since theprediction is performed in units of pixels, a structure including alinear structure formed of a first linear area or a second linear areacan be accurately detected.

In the first aspect, for example,

-   -   in the detecting, a first X-ray image may be created by        converting a resolution of the target chest X-ray image into a        first resolution, which is lower than the resolution of the        target chest X-ray image,    -   a second X-ray image may be created by converting the resolution        of the target chest X-ray image into a second resolution, which        is higher than the first resolution but equal to or lower than        the resolution of the target chest X-ray image,    -   a structure of a first size may be detected from the first X-ray        image,    -   a search area smaller than the second X-ray image may be set in        the second X-ray image on a basis of a result of the detection        of the structure of the first size, and    -   a structure of a second size, which is smaller than the first        size, may be detected in the search area.

In this aspect, a structure of a first size is detected from a firstX-ray image of a first resolution. A search area is set in a secondX-ray image of a second resolution, which is higher than the firstresolution, and a structure of a second size, which is smaller than thefirst size, is detected in the search area. According to this aspect,therefore, a search area smaller than the target chest X-ray image isset when a high-resolution image is used. As a result, the amount ofmemory used is reduced. Consequently, even when memory capacity is low,a decrease in structure detection performance can be suppressed.

In the first aspect, for example,

-   -   in the detection of the structure of the first size, an        anatomical structure may be detected from the first X-ray image        as the structure of the first size, and    -   in the detection of the structure of the second size, a linear        structure may be detected in the search area of the second X-ray        image as the structure of the second size.

According to this aspect, since the anatomical structure is of the firstsize, which is relatively large, the anatomical structure can beappropriately detected from the first X-ray image of the firstresolution, which is relatively low. In addition, since the linearstructure is of the second size, which is relatively small, the linearstructure can be appropriately detected in the search area set in thesecond X-ray image of the second resolution, which is relatively high.

In the first aspect, for example,

-   -   in the setting of the search area, the search area may be set        using a relative positional relationship between the structure        of the first size and the structure of the second size read from        a position memory storing the relative positional relationship        in advance.

According to this aspect, a position of a structure of the second sizecan be detected from a position of a structure of the first sizeobtained as a result of a first detection sub-step and a relativepositional relationship between the structure of the first size and thestructure of the second size. The structure of the second size,therefore, can be certainly detected by setting a search area such thatthe search area includes the detected position of the structure of thesecond size.

In the first aspect, for example,

-   -   in the extracting, an area obtained by expanding a contour of        the structure outward and inward by a certain number of pixels        may be determined as the neighboring area of the structure.

In this aspect, a pixel group in an area extending outside a contour ofa structure over a certain number of pixels and a pixel group in an areaextending inside the contour of the structure over the certain number ofpixels are extracted. According to this aspect, therefore, a level ofcontrast of the contour of the structure can be improved. As a result,the structures becomes easier to visually recognize.

In the first aspect, for example,

-   -   in the extracting, an area obtained by expanding the structure        outward by a certain number of pixels may be determined as the        neighboring area of the structure.

In this aspect, a pixel group in an area obtained by expanding astructure outward by a certain number of pixels is extracted. Accordingto this aspect, therefore, a level of contrast in an area larger than astructure by the certain number of pixels can be improved. As a result,the structure becomes easier to visually recognize.

In the first aspect, for example,

-   -   in the extracting, all detected structures may be used.

According to this aspect, levels of contrast in all neighboring areas ofdetected structures can be improved.

The first aspect may further include, for example,

-   -   selecting, by a user, at least one of detected structures.

In the extracting, only the at least one of the detected structuresselected by the user may be used.

According to this aspect, a level of contrast in a neighboring area of adesired structure can be improved by selecting the desired structure.

The first aspect may further include, for example,

-   -   displaying, on a display, the target chest X-ray image whose        luminance has been converted.

In the converting of the luminance, the luminance of each pixel value inthe entirety of the target chest X-ray image may be converted using thecontrast conversion expression and a tone reduction expression forreducing the tone of the target chest X-ray image.

According to this aspect, even when tone that can be displayed on adisplay is lower than that of a target chest X-ray image, the targetchest X-ray image whose level of contrast in a neighboring area of astructure has been improved can be displayed on the display with tonesuitable for the display.

A fourth aspect of the present disclosure is

-   -   a server apparatus including:    -   an obtainer that obtains a target chest X-ray image, which is a        chest X-ray image to be interpreted;    -   a detector that detects, in the target chest X-ray image using a        model obtained as a result of machine learning, a structure        including a linear structure formed of a first linear area that        has been drawn by projecting anatomical structures whose X-ray        transmittances are different from each other or a second linear        area drawn by projecting an anatomical structure including a        wall of a trachea, a wall of a bronchus, or a hair line;    -   an extractor that extracts a pixel group corresponding to a        neighboring area of the structure; and    -   an equalizer that generates a contrast conversion expression for        histogram equalization using a histogram of the pixel group; and    -   a luminance converter that converts luminance of each pixel        value in entirety of the target chest X-ray image using the        contrast conversion expression; and    -   a communication controller that transmits the target chest X-ray        image whose luminance has been converted to an external terminal        apparatus.

In the fourth aspect, a structure including a linear structure formed ofa first linear area that has been drawn by projecting anatomicalstructures whose X-ray transmittances are different from each other or asecond linear area drawn by projecting an anatomical structure includinga wall of a trachea, a wall of a bronchus, or a hair line is detected ina target chest X-ray image, which is a chest X-ray image to beinterpreted, using a model obtained as a result of machine learning. Apixel group corresponding to a neighboring area of the detectedstructure is extracted. A contrast conversion expression for histogramequalization is generated using a histogram of the extracted pixelgroup. Luminance of each pixel value in the entirety of the target chestX-ray image is converted using the generated contrast conversionexpression. The target chest X-ray image whose luminance has beenconverted is transmitted to an external terminal apparatus. According tothe fourth aspect, therefore, a user of a terminal apparatus can obtaina target chest X-ray image whose level of contrast in a neighboring areaof a structure has been improved without being affected by pixels havingpixel values whose frequencies are high.

Embodiments

Embodiments of the present disclosure will be described hereinafter withreference to the drawings. In the drawings, the same components aregiven the same reference numerals, and redundant description thereof isomitted as necessary.

First Embodiment

FIG. 1 is a block diagram schematically illustrating the configurationof an image tone conversion apparatus 100 that performs a method forconverting tone of a chest X-ray image according to a first embodiment.FIG. 2 is a block diagram schematically illustrating a networkconfiguration 410 in a medical facility,

As illustrated in FIG. 2, the network configuration 410 in the medicalfacility includes an intra network 400. The image tone conversionapparatus 100, a medical image management system 200, and a chest X-rayimage capture apparatus 300 are connected to the intra network 400. Themedical image management system 200 stores and manages chest X-rayimages, computer tomography (CT) images, magnetic resonance imaging(MRI) images, and the like. The chest X-ray image obtaining apparatus300 captures chest X-ray images of patients and persons who receive amedical examination. Chest X-ray images captured by the chest X-rayimage capture apparatus 300 are transmitted and saved to the medicalimage management system 200.

The image tone conversion apparatus 100, the medical image managementsystem 200, and the chest X-ray image capture apparatus 300 need notnecessarily be connected to the intra network 400 in the same medicalfacility. The image tone conversion apparatus 100 and the medical imagemanagement system 200 may be software operating on a server in a datacenter outside the medical facility, a private cloud server, a publiccloud server, or the like. The chest X-ray image capture apparatus 300may be installed in a hospital or a vehicle that goes around to be usedfor a medical examination or the like. As the medical image managementsystem 200, a picture archiving and communication system (PACS), forexample, is used.

As illustrated in FIG. 1, the image tone conversion apparatus 100includes a LUT storage unit 105, an image memory 106, a communicationunit 107, a display 108, a central processing unit (CPU) 120, and amemory 121. The image tone conversion apparatus 100 is achieved, forexample, by a personal computer.

The communication unit 107 communicates with the medical imagemanagement system 200 and the like over the intra network 400, The LUTstorage unit 105 is achieved, for example, by a hard disk or asemiconductor memory. The LUT storage unit 105 stores a tone conversionLUT. The image memory 106 is achieved, for example, by a hard disk or asemiconductor memory. The image memory 106 stores obtained target chestX-ray images and chest X-ray images whose luminance has been converted.The display 108 has a function of displaying 8-bit (256-tone) images inthe present embodiment. The display 108 is achieved by a liquid crystaldisplay, for example, and displays a target chest X-ray image for adoctor or a radiologist, who is a user, to give an image diagnosis orperform image checking after the image is captured. The display 108 alsodisplays chart information regarding a patient for whom the target chestX-ray image has been captured, a report input screen, on which a resultof the image diagnosis is entered, and the like.

The memory 121 is achieved, for example, by a semiconductor memory. Thememory 121 includes, for example, a read-only memory (ROM), arandom-access memory (RAM), and an electrically erasable programmableread-only memory (EEPROM). The ROM of the memory 121 stores a controlprogram for operating the CPU 120 according to the first embodiment.

The CPU 120 executes the control program according to the firstembodiment stored in the memory 121 to function as a structure detectionunit 111, a pixel extraction unit 112, a histogram calculation unit 113,a histogram equalization unit 114, a luminance conversion unit 115, adisplay control unit 116, and a communication control unit 117.

The structure detection unit 111 (an example of a detection unit)detects predefined structures from a target chest X-ray image saved inthe image memory 106. The pixel extraction unit 112 (an example of anextraction unit) extracts pixel groups corresponding to neighboringareas of the structures detected by the structure detection unit 111.The histogram calculation unit 113 calculates luminance histograms fromthe pixel groups extracted by the pixel extraction unit 112. Thehistogram equalization unit 114 performs histogram equalization usingthe luminance histograms calculated by the histogram calculation unit113. The histogram equalization unit 114 also reduces tone and obtains atone conversion LUT. The histogram equalization unit 114 stores the toneconversion LUT in the LUT storage unit 105. The luminance conversionunit 115 converts luminance of all pixels of the target chest X-rayimages using the tone conversion LUT stored in the LUT storage unit 105.The display control unit 116 displays the target chest X-ray image whoseluminance has been converted by the luminance conversion unit 115 on thedisplay 108. The communication control unit 117 (an example of anobtaining unit) controls the communication unit 107, Functions of thestructure detection unit 111, the pixel extraction unit 112, thehistogram calculation unit 113, the histogram equalization unit 114, theluminance conversion unit 115, and the display control unit 116 will bedescribed later.

FIG. 3 is a flowchart schematically illustrating a process performed bythe image tone conversion apparatus 100 according to the firstembodiment. First, in step S50, the communication control unit 117 (anexample of an obtaining unit) obtains a target chest X-ray image fromthe medical image management system 200 through the communication unit107 and saves the obtained target chest X-ray image to the image memory106. In step S100, the structure detection unit 111 reads the targetchest X-ray image from the image memory 106 and detects one or morepredefined structures from the target chest X-ray image.

Each of the one or more structures is (i) a line or an area in the chestX-ray image indicating an anatomical structure of a human body, (ii) aline or an area in the chest X-ray image indicating an area of ananatomical structure, or (iii) a boundary line in the chest X-ray imageindicating a boundary between anatomical structures whose X-raytransmittances are different from each other.

Each of the one or more structures is classified into a linear structureor an area structure. A linear structure may be a boundary line in achest X-ray image, a line in a chest X-ray image indicating ananatomical structure, or a line in a chest X-ray image indicating a partof an anatomical structure. A structure that is not a linear structure,that is, a structure that is not regarded as a line, is defined as anarea structure. Because there are linear structures wider than one pixelin images, however, linear structures and area structures can beindistinguishable from each other. For this reason, structures whoselength divided by width is equal to or larger than a threshold, forexample, may be defined as a linear structure. The threshold may be setat a value with which a structure can be regarded as a line and may be,say, 10, 100, or 1,000. FIGS. 4A to 4C and FIGS. 5A to 5C illustrateexamples of the linear structure, and FIGS. 6A to 6C illustrate anexample of the area structure.

FIG. 4A is a diagram illustrating a chest X-ray image Ix including ashadow in the descending aorta (i.e., a boundary line caused by adifference in X-ray transmittance between the descending aorta and thelung parenchyma; an example of a first linear area). FIG. 4B is adiagram illustrating a mask image Px of the shadow in the descendingaorta. FIG. 4C is a diagram illustrating an image displayed bysuperimposing the mask image Px illustrated in FIG. 4B upon the chestX-ray image Ix illustrated in FIG. 4A. FIG. 5A is a diagram illustratingthe chest X-ray image Ix including a shadow in the right dorsaldiaphragm (right dorsal lung base) (i.e., a boundary line caused by adifference in X-ray transmittance between a dorsal bottom of the lungparenchyma and ventral organs; an example of the first linear area).FIG. 5B is a diagram illustrating a mask image Py of the shadow in theright dorsal diaphragm. FIG. 5C is a diagram illustrating an imagedisplayed by superimposing the mask image Py illustrated in FIG. 5B uponthe chest X-ray image Ix illustrated in FIG. 5A, FIG. 6A is a diagramillustrating a chest X-ray image Ix including an area in which the firstthoracic vertebra is projected. FIG. 6B is a diagram illustrating a maskimage Pz of the first thoracic vertebra. FIG. 6C is a diagramillustrating an image displayed by superimposing the mask image Pzillustrated in FIG. 6B upon the chest X-ray image Ix illustrated in FIG.6A.

A mask image expresses an area of a corresponding chest X-ray imageoccupied by a structure in binary representation or grayscale. In thepresent embodiment, a binary mask image is employed. A mask image iscreated and prepared by a person with a medical background as learningdata used when the structure detection unit 111 is subjected to machinelearning. The structure detection unit 111 subjected to machine learningoutputs a mask image as a result of processing of a target chest X-rayimage.

In the present embodiment, an artificial neural network is used as meansfor performing machine learning on the structure detection unit 111.More specifically, U-Net disclosed in O. Ronneberger, P. Fischer, and T.Brox, “U-Net: Convolutional Networks for Biomedical Image Segmentation”,Medical Image Computing and Computer-Assisted Intervention (MICCAI),Springer, LNCS, Vol.9351: 234-241, 2015 is used as an artificial neuralnetwork that performs semantic segmentation for extracting a target areafrom a target image in units of pixels. “Semantic segmentation” refersto recognition of an image in units of pixels.

FIG. 7 is a diagram schematically illustrating the architecture ofU-Net. U-Net is a convolutional neural network including an encoder ECDand a decoder DCD illustrated in FIG. 7. An input image is input to aninput layer IL of U-Net, and U-Net outputs an output image to an outputlayer OL. Machine learning is performed by giving U-Net a large numberof pairs of an input image, such as those illustrated in FIGS. 4A, 5A,and 6A, and a mask image, such as those illustrated in FIGS. 4B, 5B, and6B,

More specifically, a large number of chest X-ray images Ix, such as thatillustrated in FIG. 4A, are input to U-Net, and machine learning isperformed such that U-Net outputs mask images Px, such as thatillustrated in FIG. 4B. As a result, a structure detection unit 111 fordetecting a shadow in the descending aorta is generated. In addition, alarge number of chest X-ray images Ix, such as that illustrated in FIG.5A, are input to U-Net, and machine learning is performed such thatU-Net outputs mask images Py, such as that illustrated in FIG. 5B. As aresult, a structure detection unit 111 for detecting a shadow in theright dorsal diaphragm is generated. In addition, a large number ofchest X-ray images Ix, such as that illustrated in FIG. 6A, are input toU-Net, and machine learning is performed such that U-Net outputs maskimages Pz, such as that illustrated in FIG. 6B. As a result, a structuredetection unit 111 for detecting the first thoracic vertebra isgenerated. When a target chest X-ray image is input to the structuredetection unit 111 for detecting a shadow in the descending aorta afterthe machine learning, for example, a shadow in the descending aorta isdetected as an area of a structure defined in the machine learning.

In the present embodiment, machine learning is performed on U-Nets thatdetect a total of N predefined structures (N is an integer equal to orlarger than 1) to prepare N U-Nets subjected to the machine learning.These N U-Nets subjected to the machine learning are used as thestructure detection unit 111. Alternatively, another neural network,such as one disclosed in L. Long, E. Shelhamer, and T. Darrell, “FullyConvolutional Networks for Semantic Segmentation”, CVPR, 2015, may beused instead of U-Net as an artificial neural network that performssemantic segmentation.

In step S200 illustrated in FIG. 3, the pixel extraction unit 112detects structures 0, . . . , k, . . . , and N−1. The pixel extractionunit 112 then extracts a group P₀ of pixel values of pixels included ina neighboring area R₀ of structure 0, . . . , a group P_(k) of pixelvalues of pixels included in a neighboring area R_(k) of structure k, .. . , and a group P_(N−1) of pixel values of pixels included in aneighboring area R_(N−1) of structure N−1. The group P_(k) of pixels isrepresented by expression (1). Expression (16) indicates that the groupP_(k) of pixel values is a group of pixel values p_(x, y) at coordinates(x, y) included in the neighboring area R_(k).

P _(k) ={p _(x,y) |p _(x,y) ∈R _(k)}  (1)

FIG. 8A is a diagram schematically illustrating an example of a linearstructure. FIG. 8B is a diagram schematically illustrating an example ofa neighboring area of the linear structure illustrated in FIG. 8A. FIG.9A is a diagram schematically illustrating an example of an areastructure. FIG. 9B is a diagram schematically illustrating an example ofa neighboring area of the area structure illustrated in FIG. 9A.

In FIGS. 8A and 8B, the pixel extraction unit 112 extracts a contourMLpr of a linear structure ML, which is a structure detected by thestructure detection unit 111. The pixel extraction unit 112 calculates aneighboring area Mnh1 by expanding the contour MLpr outward and inwardby a certain number of pixels through a morphological process. The pixelextraction unit 112 extracts a group of pixels values of pixels includedin the neighboring area Mnh1.

Here, a reason for expanding the contour MLpr by the certain number ofpixels will be described with reference to FIG. 19. FIG. 19(a) is anenlarged view of the linear structure ML, and FIG. 19(b) illustrates aluminance profile on a line (denoted by PF in FIG. 19(a)) across thelinear structure ML. A horizontal axis in FIG. 19(b) represents an imagespace (x coordinate) of FIG. 19(a), and a vertical axis representsluminance on the linear PF in an image illustrated in FIG. 19(a). Inmany images, adjacent pixel values smoothly change when enlarged asillustrated in FIG. 19(b) even at edges. Boundary lines of the linearstructure ML illustrated in FIG. 19(a), therefore, are recognized asMLpr1 and MLpr2 indicated in FIG. 19(b). Since the present disclosureaims to improve a level of contrast of the linear structure ML, pixelvalues (luminance values) V1 and V3 illustrated in FIG. 19(b) need to beused in subsequent histogram equalization. If pixels within ranges(denoted by Ca and Cb in FIG. 19(b)) only slightly away from theboundary lines MLpr1 and MLpr2 illustrated in FIG. 19(b) are used,however, the level of contrast of the linear structure ML is notsufficiently improved. Pixels having the pixel values V1 and V3 aretherefore used by expanding the contour MLpr1 and MLpr2 by the certainnumber of pixels. The certain number of pixels may be determined, forexample, by sequentially calculating differences between adjacent pixels(i.e., changes in luminance) near the contour MLpr in a direction thatgoes away from the contour MLpr until the changes in luminance become acertain percentage (e.g., 5% to 10%) of changes in the luminance of thelinear structure ML (|V1-V3| in FIG. 19(b)).

In FIGS. 9A and 9B, the pixel extraction unit 112 extracts a contourRGpr of an area structure RG, which is a structure detected by thestructure detection unit 111. The pixel extraction unit 112 calculates aneighboring area Rnh1 by expanding the contour RGpr outward and inwardby a certain number of pixels through a morphological process. The pixelextraction unit 112 extracts a group of pixel values of pixels includedin the neighboring area Rnh1. As described with reference to FIGS. 8A to9B, in the present embodiment, the neighboring area R_(k) is obtained byexpanding the contour RGpr of the structure RG inward and outward by thecertain number of pixels.

In step S300 illustrated in FIG. 3, the pixel extraction unit 112creates a union S of the group P₀ of pixel values, the group P_(k) ofpixel values, . . . , and the group P_(N−1) of pixel values representedby expression (2).

S=P ₀ ∪P ₁ ∪P ₂ ∪P ₃ ∪ . . . ∪P _(N−1)   (2)

Next, in step S400, the histogram calculation unit 113 creates ahistogram of the pixel values included in the union S created in stepS300. The created histogram is called a “luminance histogram”. The pixelvalues indicate luminance values,

In step S500, the histogram equalization unit 114 generates a contrastconversion expression for histogram equalization using the createdluminance histogram. A luminance value q(z) after contrast conversion isrepresented by expression (3), which is the contrast conversionexpression, using a luminance value z included in a target chest X-rayimage before the contrast conversion, a frequency H(z) of the luminancevalue z included in the union 5, the number A of elements of the union S(i.e., the number of pixels included in the union S defined byexpression (2)), and a luminance maximum value Zmax. That the frequencyH(z) is a frequency of a pixel value, that is, the luminance value z,included in the union S means that the frequency H(z) does not includethe frequency of the luminance value z outside the neighboring areas R₀to R_(N−1) in the target chest X-ray image.

In the present embodiment, for example, tone of a target chest X-rayimage before tone reduction is 12 bits (4,096 tones), and tone of thetarget chest X-ray image after the tone reduction is 8 bits (256 tones).Here, the above-described contrast conversion is performed before thetone reduction, and the luminance maximum value Zmax is 4,095.

$\begin{matrix}{{q(z)} = {\frac{Z_{\max}}{A} \cdot {\sum\limits_{i = 0}^{z}\; {H(i)}}}} & (3)\end{matrix}$

The luminance value q(z) after the histogram equalization in expression(3) is calculated for the luminance value z equal to or larger than 0but equal to or smaller than Z. When z=0, for example,

-   q(0)=H(0)Zmax/A-   When z=1, for example,-   q(1)={H(0)+H(1)}Zmax/A-   When z=2, for example,-   q(2)={H(0)+H(1)+H(2)}Zmax/A-   When z=Zmax =4,095, for example,-   q(4095)={H(0)+H(4095)}Zmax/A

In step S600, the histogram equalization unit 114 calculates an 8-bitluminance value t(z) from the 12-bit luminance value q(z) usingexpression (4), which is a tone reduction expression, to convert a12-bit image into an 8-bit image.

t(z)=q(z)/16   (4)

In expressions (3) and (4), decimals are rounded off or dropped toobtain the integral luminance values q(z) and t(z). In expression (4),therefore, the luminance value q(z) is an integer within a range of 0 to4,095, and the luminance value t(z) is an integer within a range of 0 to255.

The histogram equalization unit 114 also creates a tone conversion LUT1000 (FIG. 10). The histogram equalization unit 114 stores the createdtone conversion LUT 1000 in the LUT storage unit 105.

FIG. 10 is a diagram schematically illustrating an example of the toneconversion LUT 1000, As illustrated in FIG. 10, the original luminancevalue z and the luminance value t(z) after the histogram equalizationand the tone reduction are associated in the tone conversion LUT 1000with each other. As described above, the luminance value z is an integerwithin the range of 0 to 4,095, and the luminance value t(z) is aninteger within the range of 0 to 255.

The neighboring areas Mnh1 and Rnh1 of the structures illustrated inFIGS. 8B and 9B, respectively, include both pixels constituting thestructures and pixels that do not constitute the structures. That is,the union S includes, for each of the N structures, pixels constitutingthe structure and pixels that do not constitute the structure. As aresult, by performing histogram equalization on the luminance histogramof the union S, the tone conversion LUT 1000 for improving a level ofcontrast between each of the N structures and corresponding boundariesin the target chest X-ray image.

In step S700 illustrated in FIG. 3, the luminance conversion unit 115converts the luminance of all pixels of the chest X-ray image using thetone conversion LUT 1000 created in step S600. In step S800, the displaycontrol unit 116 displays, on the display 108, the target chest X-rayimage converted into the 8-bit image as a result of the tone conversion.Luminance conversion for improving levels of contrast of all the Nstructures and tone conversion for reducing tone of all the N structuresare thus performed. The target chest X-ray image converted into the8-bit image as a result of the tone conversion is thus displayed on thedisplay 108.

Definitions of terms will be described hereinafter. “Tone conversion”refers to luminance conversion including both (A) contrast conversionfor improving a level of contrast of an image and (B) tone reduction forconverting (reducing) the number of tones for expressing gradation in animage. Histogram equalization and gamma correction are a specificexample of a method used in (A) contrast conversion. “Luminanceconversion”, on the other hand, does not refer to a specific conversionprocess but simply refers to conversion of luminance (pixel values).“Tone” herein refers to “gradation in an image” in a broad sense and“the number of shades in a digital image” (e.g., 256 tones) in a narrowsense. A pixel value may indicate a luminance value.

Although the neighboring area R_(k) is obtained by expanding the contourRGpr of the structure RG inward and outward by the certain number ofpixels in step S200 in the present embodiment, the neighboring areaR_(k) is not limited to this.

FIG. 11A is a diagram schematically illustrating another example of theneighboring area of the linear structure illustrated in FIG. 8A. FIG.11B is a diagram schematically illustrating another example of theneighboring area of the area structure illustrated in FIG. 9A.

In the examples illustrated in FIGS. 11A and 11B, the pixel extractionunit 112 determines areas obtained by expanding the linear structure MLand the area structure RG outward by a certain number of pixels asneighboring areas Mnh2 and Rnh2, respectively. The neighboring areasMnh2 and Rnh2 of the structures ML and RG illustrated in FIGS. 11A and11B include both pixels constituting the structures and pixels outsidethe structures. The number of pixels constituting the structures,however, is larger than in the case of the neighboring areas Mnh1 andRnh1 illustrated in FIGS. 8B and 9B. A tone conversion LUT for improvinga level of contrast within a structure and a level of contrast betweenthe structure and corresponding boundaries can therefore be obtained forall the N structures by performing histogram equalization on a luminancehistogram of a union S of pixels included in the neighboring areas Mnh2and Rnh2 illustrated in FIGS. 11A and 11B. When a structure is a bonesuch as a rib or a collarbone, for example, a level of contrast oftrabecula improves.

As described above, according to the first embodiment of the presentdisclosure, a structure including a linear structure formed of a firstlinear area drawn by projecting anatomical structures whose X-raytransmittances are different from each other or a second linear areadrawn by projecting an anatomical structure including a wall of atrachea, a wall of a bronchus, or a hair line is detected. A toneconversion LUT is obtained by generating a contrast conversionexpression for histogram equalization using a histogram of a group ofpixels values of pixels corresponding to a neighboring area of thedetected structure and a tone reduction expression for reducing tone.Luminance of the entirety of a target chest X-ray image is convertedusing the tone conversion LUT. As a result, tone conversion forimproving a level of contrast of a structure important in making adiagnosis can be performed without being affected by pixels havingluminance values whose frequencies are high.

Second Embodiment

FIG. 12 is a block diagram schematically illustrating the configurationof an image tone conversion apparatus 100A that performs a method forconverting tone of a chest X-ray image according to a second embodiment.Unlike the image tone conversion apparatus 100 illustrated in FIG. 1,the image tone conversion apparatus 100A illustrated in FIG. 12 newlyincludes a normal model storage unit 103 and also includes a CPU 120Ainstead of the CPU 120 and a memory 121A instead of the memory 121.

The normal model storage unit 103 (an example of a position memory)stores information regarding relative positional relationships betweenstructures in advance. The memory 121A is configured in the same manneras the memory 121, and includes, for example, a ROM, a RAM, and anEEPROM. The ROM of the memory 121A stores a control program foroperating the CPU 120A according to the second embodiment.

The CPU 120A executes the control program according to the secondembodiment stored in the memory 121A to function as the structuredetection unit 111, the pixel extraction unit 112, the histogramcalculation unit 113, the histogram equalization unit 114, the luminanceconversion unit 115, the display control unit 116, a resolutionconversion unit 109, and a search area setting unit 110.

The resolution conversion unit 109 creates images having differentresolutions by performing reduction conversion of more than one stageson a target chest X-ray image. The resolution conversion unit 109 storesthe created images in the image memory 106. The search area setting unit110 sets an area to be searched for a structure in an image of a higherresolution using a result of detection of a structure performed by thestructure detection unit 111 on a low-resolution image and theinformation regarding relative positional relationships betweenstructures stored in the normal model storage unit 103.

Next, a process performed by the image tone conversion apparatus 100Aaccording to the second embodiment will be described. The overallprocess is the same as in the first embodiment described with referenceto the flowchart of FIG. 3.

FIG. 13 is a flowchart schematically illustrating a process performed bythe image tone conversion apparatus 100A according to the secondembodiment in step S100 (FIG. 3). FIG. 14 is a diagram schematicallyillustrating resolution information 2600.

In step S141 illustrated in FIG. 13, the resolution conversion unit 109creates R (e.g., R=3 in the present embodiment) different reduced imagesfor the target chest X-ray image obtained in step S50 (FIG. 3). Theresolution of a chest X-ray image is usually 2,000 to 3,000 pixels eachside. In the second embodiment, the resolution of the target chest X-rayimage is, for example, 2,048×2,048. Resolutions of three differentreduced images created by the resolution conversion unit 109 are, forexample, 1,024×1,024, 512×512, and 256×256.

In the second embodiment, resolution i is set at 0, 1, 2, and 3 for theimages in ascending order of resolution. That is, the resolution i ofthe 256×256 image is 0, the resolution i of the 512×512 image is 1, theresolution i of the 1,024×1,024 image is 2, and the resolution i of the2,048×2,048 image (i.e., the original image) is 3. The resolutionconversion unit 109 stores the created low-resolution reduced images inthe image memory 106.

Next, in step S102, the structure detection unit 111 reads the imagewhose resolution i=0 (i.e., the lowest-resolution, namely, 256×256,image) from the image memory 106 as a structure detection target image.Next, in step S103, the structure detection unit 111 detects structuresassociated with the image of the resolution i (the image whoseresolution i=0 in a first round of step S103) on the basis of theresolution information 2600 (FIG. 14).

As illustrated in FIG. 14, the resolution information 2600 includes astructure identifier (ID) field 2601 and a resolution i field 2602. Inthe structure ID field 2601, N structures whose structure IDs are 0 toN−1 and that are defined in the first embodiment are set. In theresolution i field 2602, the resolution of an image to be used to detecta corresponding structure in the structure ID field 2601 is defined. Astructure whose structure ID is 0, for example, is detected from theimage whose resolution i is 0, that is, the 256×256 image. Although onlyone resolution is set for each structure in FIG. 14, the number ofresolutions is not limited to this. For example, two or more resolutionmay be set depending on a structure, and the structure may be detectedusing images of the two or more resolutions.

As in the first embodiment, the structure detection unit 111 detects astructure using U-Net disclosed in “U-Net: Convolutional Networks forBiomedical Image Segmentation”, As described above, U-net is a type ofconvolutional neural network. A convolutional neural network is a typeof deep neural network. A neural network including two or moreintermediate layers is called a deep neural network. During machinelearning for a deep neural network and detection of a structure,processing speed is usually increased using a graphics processing unit(GPU). At this time, it might be difficult to handle a high-resolutionimage due to a restriction to the memory capacity of the GPU. In such acase, an image obtained by reducing an original image and decreasing theresolution of the original image is input to U-Net. In this case,however, detection performance for small structures, such as linearstructures, can decrease. For this reason, in the second embodiment, thestructure detection unit 111 detects a relatively large (an example of afirst size) structure from a low-resolution image and a relatively small(an example of a second size) structure within a limited search area bytrimming a high-resolution image.

In step S104 illustrated in FIG. 13, the structure detection unit 111increments the resolution i. In a first round of step S104, i=1, In stepS105, the structure detection unit 111 determines whether the resolutioni has exceeded an upper limit (i.e., i=R+1) of resolution, If theresolution i has exceeded the upper limit of resolution (YES in stepS105), the process illustrated in FIG. 13 ends, and step S100 (FIG. 3)ends. If the resolution i has not exceeded the upper limit of resolution(NO in step 3105), the process proceeds to step S106.

In step S106, the search area setting unit 110 selects all structuresassociated with the resolution i (i=1 in a first round of step S106) onthe basis of the resolution information 2600 illustrated in FIG. 14 andsets a search area for an image of the resolution i for each of thecorresponding structures. Relative positional relationships betweenstructures are obtained from a large number of binary mask images ofstructures, such as those illustrated in FIGS. 4B, 5B, and 6B and savedto the normal model storage unit 103 in advance. The search area settingunit 110 reads the relative positional relationships saved in the normalmodel storage unit 103 and uses the relative positional relationships toset search areas. After step S106 ends, the process returns to step3103, In a second round of step S103, the structure detection unit 111detects structures associated with the image of the resolution i (theimage whose resolution i=1 in the second step S103) on the basis of theresolution information 2600 (FIG. 14). Steps S103 to S106 are thenrepeated while the resolution i does not exceed the upper limit ofresolution (NO in step S105),

FIG. 15 is a diagram schematically illustrating steps S103 to S106illustrated in FIG. 13, In FIG. 15, first, the structure detection unit111 detects structures Pa and Pb from a chest X-ray image la whoseresolution is low (i=0) (step 3103). In the example illustrated in FIG.15, the structure Pa is the right lung field, and the structure Pb isthe left lung field. In the present embodiment, the chest X-ray image lais an example of a first X-ray image, the resolution i=0 (256×256) is anexample of a first resolution, and the size of the structures Pa and Pbis an example of the first size.

Next, the resolution i is incremented (step S104), and the search areasetting unit 110 sets a search area in a chest X-ray image Ib whoseresolution is intermediate (i=1) (step S106). Although FIG. 15illustrates only a search area SA1, a search area is set in step S106for each of the structure IDs associated with the resolution i. Eachsearch area is set using a structure to be detected indicated by astructure ID and a positional relationship between already detectedstructures (the structures Pa and Pb in the example illustrated in FIG.15) saved in the normal model storage unit 103,

Next, the structure detection unit 111 detects a structure in the searcharea of the chest X-ray image Ib of the intermediate resolution (i=1)(step S103). Although FIG. 15 illustrates only a structure Pc detectedin the search area SA1, a structure to be detected is detected in stepS163 in each search area. In the present embodiment, the chest X-rayimage Ib is an example of a second X-ray image, the resolution i=1(512×512) is an example of a second resolution, and the size of thestructure Pc is an example of the second size.

Next, the resolution i is incremented (step S104), and the search areasetting unit 110 sets a search area in a chest X-ray image Ic whoseresolution is high (i=2) (step S106). Although FIG. 15 illustrates onlya search area SA2, a search area is set in step S106 for each of thestructure IDs associated with the resolution i. Each search area is setusing a structure to be detected indicated by a structure ID and apositional relationship between already detected structures (thestructures Pa and Pb in the example illustrated in FIG. 15) saved in thenormal model storage unit 103.

Next, the structure detection unit 111 detects a structure in the searcharea of the chest X-ray image Ic of the high resolution (i=2) (stepS103). Although FIG. 15 illustrates only a structure Pd detected in thesearch area SA2, a structure to be detected is detected in step S103 ineach search area.

As described above, according to the second embodiment of the presentdisclosure, when a deep neural network such as U-Net is used as thestructure detection unit 111, a decrease in structure detectionperformance can be suppressed since a search area smaller than a targetchest X-ray image is set when a high-resolution image is used even ifthe memory capacity of the GPU is low.

Furthermore, tone conversion for improving a level of contrast of astructure important in making a diagnosis can be performed without beingaffected by pixels having luminance values whose frequencies are high,which is the effect produced by the first embodiment.

Third Embodiment

FIG. 16 is a block diagram schematically illustrating the configurationof a tone conversion apparatus 144E that performs a method forconverting tone of a chest X-ray image according to a third embodiment.Unlike the image tone conversion apparatus 100 illustrated in FIG. 1,the tone conversion apparatus 1468 illustrated in FIG. 16 newly includesan input unit 118 and also includes a CPU 120B instead of the CPU 120and a memory 121B instead of the memory 121.

The input unit 118 is operated by a user such as a doctor or aradiologist. The memory 121B is configured in the same manner as thememory 121 and includes, for example, a ROM, a RAM, and an EEPROM. TheROM of the memory 121B stores a control program for operating the CPU120B according to the third embodiment.

The CPU 120B executes the control program according to the thirdembodiment stored in the memory 121B to function as the structuredetection unit 111, a pixel extraction unit 112B, the histogramcalculation unit 113, the histogram equalization unit 114, the luminanceconversion unit 115, the display control unit 116, and the communicationcontrol unit 117.

The pixel extraction unit 112 according to the first embodiment extractspixel values of pixels corresponding to neighboring areas of all the Nstructures detected by the structure detection unit 111. The pixelextraction unit 112E according to the third embodiment, on the otherhand, pixel values of pixels corresponding to neighboring areas ofstructures selected by the user using the input unit 118 among the Nstructures detected by the structure detection unit 111.

FIG. 17 is a flowchart schematically illustrating a process performed bythe tone conversion apparatus 100B according to the third embodiment.Steps S50 and S100 illustrated in FIG. 17 are the same as thoseillustrated in FIG. 3. In step S150, which follows step S100, the pixelextraction unit 112B selects structures specified using the input unit118 among the N structures detected by the structure detection unit 111.In step S250, the pixel extraction unit 112B extracts a group of pixelvalues corresponding to a neighboring area of each of the selectedstructures, Steps S300 to S800 illustrated in FIG. 17 are the same asthose illustrated in FIG. 3.

According to the third embodiment, tone conversion for improving levelsof contrast of structures desired by the user can be performed,

Fourth Embodiment

FIG. 18 is a block diagram schematically illustrating a networkconfiguration 410A in a medical facility according to a fourthembodiment, As illustrated in FIG. 18, a server apparatus 500, a displaycontrol apparatus 600, the medical image management system 200, and thechest X-ray image capture apparatus 300 are connected to an intranetwork 400 in the medical facility in the fourth embodiment.

The server apparatus 500, the display control apparatus 600, the medicalimage management system 200, and the chest X-ray image capture apparatus300 need not necessarily be connected to the intra network 400 in asingle medical facility. The display control apparatus 600 and themedical image management system 200 may be software that operates on aserver in a data center outside the medical facility, a private cloudserver, a public cloud server, or the like, instead.

As illustrated in FIG. 18, the server apparatus 500 includes the LUTstorage unit 105, the image memory 106, the communication unit 107, aCPU 130, and a memory 131. The memory 131 is achieved, for example, by asemiconductor memory. The memory 131 includes, for example, a ROM, aRAM, and an EEPROM. The ROM of the memory 131 stores a control programfor operating the CPU 130.

The CPU 130 executes the control program stored in the memory 131 tofunction as the structure detection unit 111, the pixel extraction unit112, the histogram calculation unit 113, the histogram equalization unit114, the luminance conversion unit 115, and a communication control unit117A. The communication control unit 117A obtains a target chest X-rayimage whose luminance has been converted by the luminance conversionunit 115 to the display control apparatus 600 through the communicationunit 107.

The display control apparatus 600 (an example of a terminal apparatus)is achieved, for example, by a tablet computer and carried by a medicalworker such as a doctor or a radiologist. As illustrated in FIG. 18, thedisplay control apparatus 600 includes a CPU 140, a memory 141, an imagememory 142, a communication unit 143, and the display 108.

The memory 141 is achieved, for example, by a semiconductor memory. Thememory 141 includes, for example, a ROM, a RAM, and an EEPROM. The ROMof the memory 141 stores a control program for operating the CPU 140.The CPU 140 executes the control program stored in the memory 141 tofunction as the display control unit 116 and a communication controlunit 117B.

The communication control unit 1178 receives, through the communicationunit 143, data regarding a target chest X-ray image whose luminance hasbeen converted and that has been transmitted from the server apparatus500 and stores the received data in the image memory 142. The displaycontrol unit 116 displays, on the display 108, the target chest X-rayimage whose luminance has been converted and that is stored in the imagememory 142.

According to the fourth embodiment, the same effect as that produced bythe first embodiment can be produced. Alternatively, the CPU 130 of theserver apparatus 500 may function as the structure detection unit 111,the pixel extraction unit 112, the histogram calculation unit 113, thehistogram equalization unit 114, the luminance conversion unit 115, thecommunication control unit 117, the resolution conversion unit 109 (FIG.12), and the search area setting unit 110 (FIG. 12). In this case, thesame effect as that produced by the second embodiment can be produced.

The present disclosure can be used in diagnosis aiding systems for chestX-ray images to be interpreted and interpretation education systems formedical students or interns.

What is claimed is:
 1. A method for converting tone of a chest X-rayimage, the method being performed by a computer of an image toneconversion apparatus that converts tone of a target chest X-ray image,which is a chest X-ray image to be interpreted, the method comprising:obtaining the target chest X-ray image; detecting, in the target chestX-ray image using a model obtained as a result of machine learning, astructure including a linear structure formed of a first linear areathat has been drawn by projecting anatomical structures whose X-raytransmittances are different from each other or a second linear areadrawn by projecting an anatomical structure including a wall of atrachea, a wall of a bronchus, or a hair line; extracting a pixel groupcorresponding to a neighboring area of the structure; generating acontrast conversion expression for histogram equalization using ahistogram of the pixel group; and converting luminance of each pixelvalue in entirety of the target chest X-ray image using the contrastconversion expression.
 2. The method according to claim 1, wherein themodel obtained as a result of the machine learning is a model subjectedto the machine learning such that the structure is detected in alearning chest X-ray image, which is a chest X-ray image in a normalstate, using a neural network that performs prediction in units ofpixels.
 3. The method according to claim 2, wherein, in the detecting, afirst X-ray image is created by converting a resolution of the targetchest X-ray image into a first resolution, which is lower than theresolution of the target chest X-ray image, wherein a second X-ray imageis created by converting the resolution of the target chest X-ray imageinto a second resolution, which is higher than the first resolution butequal to or lower than the resolution of the target chest X-ray image,wherein a structure of a first size is detected from the first X-rayimage, wherein a search area smaller than the second X-ray image is setin the second X-ray image on a basis of a result of the detection of thestructure of the first size, and wherein a structure of a second size,which is smaller than the first size, is detected in the search area. 4.The method according to claim 3, wherein, in the detection of thestructure of the first size, an anatomical structure is detected fromthe first X-ray image as the structure of the first size, and wherein,in the detection of the structure of the second size, a linear structureis detected in the search area of the second X-ray image as thestructure of the second size.
 5. The method according to claim 3,wherein, in the setting of the search area, the search area is set usinga relative positional relationship between the structure of the firstsize and the structure of the second size read from a position memorystoring the relative positional relationship in advance.
 6. The methodaccording to claim 1, wherein, in the extracting, an area obtained byexpanding a contour of the structure outward and inward by a certainnumber of pixels is determined as the neighboring area of the structure.7. The method according to claim 1, wherein, in the extracting, an areaobtained by expanding the structure outward by a certain number ofpixels is determined as the neighboring area of the structure.
 8. Themethod according to claim 1, wherein, in the extracting, all detectedstructures are used.
 9. The method according to claim 1, furthercomprising: selecting, by a user, at least one of detected structures,wherein, in the extracting, only the at least one of the detectedstructures selected by the user is used.
 10. The method according toclaim 1, further comprising: displaying, on a display, the target chestX-ray image whose luminance has been converted, wherein, in theconverting of the luminance, the luminance of each pixel value in theentirety of the target chest X-ray image is converted using the contrastconversion expression and a tone reduction expression for reducing thetone of the target chest X-ray image.
 11. A storage medium storing aprogram for causing a computer of an image tone conversion apparatusthat converts tone of a target chest X-ray image, which is a chest X-rayimage to be interpreted, to perform a process, the storage medium beingnonvolatile and computer-readable, the process comprising: obtaining thetarget chest X-ray image; detecting, in the target chest X-ray imageusing a model obtained as a result of machine learning, a structureincluding a linear structure formed of a first linear area that has beendrawn by projecting anatomical structures whose X-ray transmittances aredifferent from each other or a second linear area drawn by projecting ananatomical structure including a wall of a trachea, a wall of abronchus, or a hair line; extracting a pixel group corresponding to aneighboring area of the structure; generating a contrast conversionexpression for histogram equalization using a histogram of the pixelgroup; and converting luminance of each pixel value in entirety of thetarget chest X-ray image using the contrast conversion expression. 12.An image tone conversion apparatus comprising: an obtainer that obtainsa target chest X-ray image, which is a chest X-ray image to beinterpreted; a detector that detects, in the target chest X-ray imageusing a model obtained as a result of machine learning, a structureincluding a linear structure formed of a first linear area that has beendrawn by projecting anatomical structures whose X-ray transmittances aredifferent from each other or a second linear area drawn by projecting ananatomical structure including a wall of a trachea, a wall of abronchus, or a hair line; an extractor that extracts a pixel groupcorresponding to a neighboring area of the structure; an equalizer thatgenerates a contrast conversion expression for histogram equalizationusing a histogram of the pixel group; and a luminance converter thatconverts luminance of each pixel value in entirety of the target chestX-ray image using the contrast conversion expression.
 13. A serverapparatus comprising: an obtainer that obtains a target chest X-rayimage, which is a chest X-ray image to be interpreted; a detector thatdetects, in the target chest X-ray image using a model obtained as aresult of machine learning, a structure including a linear structureformed of a first linear area that has been drawn by projectinganatomical structures whose X-ray transmittances are different from eachother or a second linear area drawn by projecting an anatomicalstructure including a wall of a trachea, a wall of a bronchus, or a hairline; an extractor that extracts a pixel group corresponding to aneighboring area of the structure; and an equalizer that generates acontrast conversion expression for histogram equalization using ahistogram of the pixel group; and a luminance converter that convertsluminance of each pixel value in entirety of the target chest X-rayimage using the contrast conversion expression; and a communicationcontroller that transmits the target chest X-ray image whose luminancehas been converted to an external terminal apparatus.
 14. A conversionmethod comprising: obtaining an X-ray image; detecting a linear area inthe X-ray image; determining a neighboring area of the linear area;providing a conversion expression, first pixel values of first pixelsbeing included in the linear area or the neighboring area of the lineararea; and converting the first pixel values and second pixel values ofsecond pixels into resulting values using the conversion expression, thesecond pixels including (i) pixels that are included in the X-ray imagebut that are not included in the linear area and (ii) pixels that areincluded in the X-ray image but that are not included in the neighboringarea; wherein the conversion expression is $\begin{matrix}{{q(z)} = {\frac{Z_{\max}}{A} \cdot {\sum\limits_{i = 0}^{z}\; {H(i)}}}} & (3)\end{matrix}$ where z is a pixel value included in the X-ray image,where q(z) is a resulting value of the pixel value z included in theresulting values, where H(i) is a number of pixels whose pixel valuesare i among the first pixel values, where Zmax is a maximum pixel valueof each of the pixels of the X-ray image, and where A is a number offirst pixels.